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SECTION  I 
INTRODUCTION 


TRAINING  SYSTEMS  EVALUATION 

Although  the  methodology  that  is  now  employed  throughout  Naval  aviation 
for  instructional  systems  development  (ISD)  has  become  sufficiently  advanced 
^  1nte9rated  format  of  a  systematic  model,  1,2  the  closely  allied 
methodology  of  training  systems  evaluation  has  not  been  formulated  comparably, 
witll  recently,  such  evaluation  was  regarded  as  merely  preparatory  activities 
necessary  to  justify  initiation  of  an  ISD  project.  Experience  gained  through 
nunerous  applications  of  the  ISD  process,  however,  has  clearly  dispelled  this 
view.  It  Is  now  apparent  that  both  the  direction  and  scope  of  an  ISD  appli¬ 
cation  depend  heavily  on  the  data,  assessments,  and  plans  that  result  from  mi 
appropriately  conducted  training  system  evaluation. 


The  training  systems  evaluation  process  developed  at  the  Naval  Training 
Equipment  Center  Is  currently  required  as  a  prerequisite  to  Initiation  of  any 
ISD  project.  The  process  Is  divided  Into  two  main  spheres  of  effort  — 
problem  analysis  and  project  planning.  Each  area  of  activity  and  its  output 
Is  described  generally  in  the  Introductory  paragraphs  that  follow,  and  In 
greater  detail  In  siirsequent  sections  of  this  report. 


OBJECTIVES  OF  PROBLEM  AMLYSIS 


While  the  extensiveness  of  any  given  problem  analysis  (PA)  may  vary 
according  to  the  complexity  and  seriousness  of  the  training  problon  It 
addresses,  the  overall  purpose  of  the  PA  Is  always  to  establish  an  evaluative 
foundation  upon  which  can  be  built  a  plan  of  action  to  solve  a  training 
problem.  Whether  solution  of  the  problem  requires  remedial  enhancement  of  an 
existing  training  program  (TP),  or  original  development  of  a  new  TP,  It  Is 
the  PA  which  provides  the  data-base  and  analytical  framework  that  governs 
the  specific  direction  taken. 

Four  primary  PA  objectives  may  be  distinguished.  They  are: 

a.  To  collect,  organize,  and  analyze  all  factual  information  necessary 
to  docunent  the  nature  of  the  training  problem.  Its  origins,  and  the  factors 
that  sustain  It. 

b.  To  evaluate  the  training  problem  In  terms  of  modern  Instructional 
systems  development  (ISD)  methodology  as  a  means  of  determining  the  training, 
managerial,  and  resource  requirements  that  will  be  needed  to  carry  out  any 
new  TP  development  or  enhancement. 


2*  — ■  i J  1977)  Training  Requirements  for  Aviation 

Weapon  systems,  "NAVTRAEQU IPCEN  Specification  MIL-T-29053,  draft. 


2.  Courseware,  Inc.  (June,  1977)  "Fleet  Aviation  Instructional  Systems 
Development  Model",  Courseware  Technical  Report,  Vols.  I-VII. 
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c.  To  access  the  feasibility  of  alternative  approaches  to  solution  of 
the  training  problem  considering  the  resources  available  for  TP  development. 
Implementation,  and  continued  operation  as  compared  with  the  constraints, 
costs,  and  utility  Inherent  In  each  approach. 

d.  To  anticipate  problem  areas  and  support  requirements  that  are  likely 
to  arise  at  some  future  time  during  the  life-qycle  of  the  revised  or  newly 
developed  training  program. 

PRELIMINARIES  TO  PROBLEM  ANALYSIS 

The  events  preceding  initiation  of  a  problem  analysis  must  achieve  three 
main  goals: 

a.  Authentication  of  the  existence  of  an  Identified  training  problem. 

b.  Justification  of  funding  for  the  proposed  problem  analysis. 

c.  Establishment  of  lines  of  communication  and  responsibility  among 
the  major  participants  In  the  project. 

The  preliminary  process  may  be  Initiated  by  the  identification  of  a  training 
problem  existing  within  an  on-going  training  program,  or  by  the  emergence  of 
a  new  weapon  system  (or  system  component)  which  Imposes  training  support 
requirements  not  present  In  the  fleet. 

Assuming  the  training  problem  arises  within  the  context  of  an  existing 
weapon  system,  the  general  flow  of  Inputs  and  actions  through  the  Nayy 
hierarchy  Is  as  follows.  After  a  training  problem  has  been  formal lv 
Identified  by  a  Navy  cormand  (squadron,  wing,  type  commander  (TYCOM), 

Chief  Naval  Operations  (CNO),  Naval  Air  Systems  Conmand  (NAVAIRSYSCOM) ,  or 
a  Navy  laboratory),  the  usual  procedure  Is  for  the  CNO  to  task  the  TYCOM 
to  conduct  an  Initial  Problan  Identification  (IPI)  and  to  prepare  an  IPI 
report.  This  report  tentatively  docunents  the  nature,  scope,  and  criti¬ 
cality  of  the  problan.  If  a  significant  training  problem  (one  which  Is 
likely  to  require  a  substantial  expenditure  of  resources  to  analyze  and 
correct)  Is  found  to  exist,  an  Operational  Requirement  for  Revision  of 
Existing  Training  System  (OR)  Is  generated  either  by  thiTChO,  NaVAIrSYSCOM, 
or  NAVTKAftJU IPCEn.  this  OR  serves  as  the  basis  for  justifying  the  funding 
necessary  to  study  the  training  problem  In  depth. 

On  approval  of  the  OR  by  the  CNO,  he  then  tasks  the  NAVAIRSYSCOM  to 
conduct  a  detailed  Problan  Analysis  (PA)  and  to  develop  a  Program  Master 
(PMP).  The  TYCOM  and  tlAVT&AEQUlhlBl  are  Informed  of  this  tasking  so  that 
they  may  prepare  to  participate  fully  In  the  effort.  Eventually, 

NAVTRAEQU IPCEN  Is  tasked  to  plan  the  PA  and  PMP,  and  communication  begins 
among  the  various  concerned  Navy  organizations  and  Independent  contractors. 
The  ensuing  comnunlcatlon  system  serves  to  Inform  the  fleet  participants 
of  the  nature  of  the  PA,  to  delineate  the  respective  roles  of  each  parti¬ 
cipant  In  the  PA  effort,  and  to  coordinate  the  resources  necessary  for  the 
PA  process. 
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Once  the  lines  of  comnunl cation  have  been  established,  a  coordination 
meeting  Is  scheduled  for  the  participants.  This  corresponds  In  time  with 
the  completion  of  the  Problem  Analysis  Plan  Report  by  NAVTRAEQU IPCEN  and  Its 
approval  by  NAVAIRSYSCUFC  TRis  plan  will  guide  the  management  and  coordi¬ 
nation  system  necessary  to  carry  out  the  PA.  Also,  by  now  the  funding 
requirements  for  the  PA  will  have  been  established  and  approved  by 
NAVAIRSYSCOM.  Collection  of  the  data  (usually  by  an  independent  contractor 
with  expertise  In  Instructional  systems  development)  needed  for  an  evaluation 
of  the  training  problem  Is  then  begun.  Upon  completion  of  data  collection  and 
analysis,  the  complete  Problem  Analysis  Report  Is  prepared  and  submitted  to 
NAVAIRSYSCOM  for  review. 

OVERVIEW  OF  THE  PROBLEM  AfWLYSIS  PROCESS 

The  extensiveness  of  the  PA  process  Is  largely  determined  by  the  nature 
of  the  training  program  (TP),  if  one  exists.  In  which  the  problem  of  concern 
has  arisen.  That  the  PA  process  should  be  keyed  more  directly  to  the  program 
than  to  the  problem  Is  but  an  extension  of  the  principles  Implicit  In  a  system 
approach  to  Instructional  development.  From  this  point  of  view,  a  TP  Is  a 
functioning  system  and  the  outputs  are  complex  products  of  the  Interactions 
among  all  the  system's  components.  A  training  problem  arising  within  an 
existing  TP  Is  regarded  as  a  kind  of  system  failure  or  malfunction  where  the 
origins  can  be  discovered  most  satisfactorily  by  an  analysis  of  the  Inter¬ 
dependent  contributions  of  component  processes  to  the  operations  of  the  system 
as  a  whole.  Hence,  It  Is  the  conditions  responsible  for  the  training  problem 
that  the  PA  seeks  to  determine,  and  it  Is  the  nature  of  these  conditions  the 
governs  the  degree  to  which  the  analysis  must  be  carried. 

The  PA  may  be  performed  either  on  an  existing  TP  In  need  of  revision, 
or  on  an  emerging  weapon  system  for  which  no  TP  has  yet  been  developed.  In 
the  latter  case,  the  PA  will  be  a  mini  ISD  project  aimed  at  an  Identification 
of  the  major  tasks  required  for  operation  and/or  maintenance  of  the  new  system. 
Since  competent  performance  of  these  tasks  Is  what,  ultimately,  the  TP  nust 
be  designed  +  produce,  the  tasks  will  define  the  primary  goals  to  be  achieved 
by  thr  TP.  once  the  task-oriented  goals  of  the  program  have  been  established, 
the  next  stage  of  the  PA  process  Is  to  determine  the  nature  of  the  ISD  project 
needed  to  generate  a  TP  that  will  attain  most  effectively  the  stated  goals. 

The  various  steps  that  will  be  required  to  carry  out  the  ISD  project  may  now 
be  enumerated,  and  from  these  are  determined  the  personnel,  facilities,  equip¬ 
ment,  time,  and  funding  requirements  necessary  for  the  Instructional  develop¬ 
ment  project.  These  are  the  main  elements  of  the  PA  process  as  It  is  applied 
to  the  problem  of  a  newly  emerging  weapon  system  for  which  no  previous  TP  has 
been  developed. 

In  the  case  of  an  existing  training  program,  the  extensiveness  of  the  PA 
usually  will  depend  on  whether  the  TP  was  originally  designed  according  to 
the  basic  principles  Intrinsic  In  ISO,  and  whether  the  original  TP  has  been 
systematically  reevaluated  and  updated  since  the  time  of  Its  Implementation. 
Should  both  of  these  conditions  be  satisfied,  the  PA  would  be  a  straight¬ 
forward,  If  not  routine,  review  of  data  pertaining  to  any  problematic  aspects 
of  the  program,  or  to  aqy  newly  required  program  changes.  However,  the  usual 
state  of  affairs  necessitating  a  PA  probably  would  be  a  training  program  that 
has  been  neither  updated  In  keeping  with  weapon  system  changes,  nor  origi¬ 
nally  designed  In  accordance  with  the  principles  of  ISD.  In  this  instance. 
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a  full-scale  PA  effort  would  be  needed  In  order  to  adequately  assess  the 
problematic  nature  of  the  existing  TP.  and  to  provide  the  data  necessary  for 
development  of  a  program  master  plan  ( PMP) ,  the  basic  work  plan  which  guides 
and  coordinates  development  of  the  revised  Instructional  system. 

A  full-scale  PA  of  an  existing  training  program  Involves  extensive 
Interviews  with  cognizant  Navy  personnel,  surveys  of  relevant  technical 
publications,  etc.,  and  first-hand  examinations  of  all  training  facilities, 
materials,  and  procedures.  Aspects  of  the  existing  TP  that  are  specifically 
examined  include  the  purpose  and  scope  of  the  program.  Its  management 
structure  and  Instructor  profile,  the  student  entry-levels,  flow-rates,  and 
evaluation  techniques;  the  training  schedule,  syllabi,  media,  equipment, 
and  aircraft;  as  well  as  training  facilities  (classrooms,  student  learning 
centers,  etc.)  and  any  other  supporting  resources.  Throughout  this  program 
review,  an  attempt  Is  made  to  Identify  any  problem  areas  (training  defici¬ 
encies,  resource  Inadequacies,  etc.)  and  factors  requiring  program  changes 
(weapons  system  changes,  new  mission  requirements,  revised  student  flow-rate, 
etc.)  that  are  likely  to  Impact  upon  subsequent  application  of  the  ISO 
process  to  TP  enhancement  or  revision.  These  data  not  only  define  the  nature 
of  the  training  problem,  but  they  also  form  the  necessary  basis  for  ISD 
project  procurement. 

FROM  ANALYSIS  TO  PROGRAM  MASTER  PLAN  (PMP) 

How  that  the  nature  of  the  training  problem,  the  conditions  giving  rise 
to  It,  and  the  resource  requirements  needed  to  meet  the  desired  training 
goals  have  been  Identified  In  the  PA  and  reviewed  for  feasibility  by 
NAVAIRSYSCOM,  the  stage  has  been  set  for  development  of  the  PMP,  the  plan 
which  will  control  all  subsequent  ISD  efforts.  The  major  Items  of  Informa¬ 
tion  contained  In  the  PMP  are: 

a.  A  detailed  listing  of  the  resources  necessary  for  support  of  the  ISD 
project,  as  well  as  their  resources. 

b.  A  statement  of  the  specific  objectives  to  be  achieved  by  the  ISD 
project. 

c.  An  enumeration  of  the  main  tasks  to  be  performed  during  the  ISD 
project,  their  time-phasing  and  scheduling  Into  the  major  milestones  to  be 
reached  throughout  the  project,  and  the  method  to  be  used  In  tracking 
progress  toward  their  completion. 

d.  A  delineation  of  the  roles,  areas  of  responsibility,  management 
systems  and  channels  of  conmunlcatlon  that  will  exist  among  participating 
Navy  organizations  and  Independent  contractors  during  the  project. 

e.  A  statement  of  the  procurement  strategy  options  and  Instrunents  that 
may  be  selected  for  use,  given  the  availability  of  funding,  for  each  activity 
and  material  requirements  encompassed  by  the  project. 

The  primary  foundation  upon  which  the  above  PMP  contents  are  based 
is  the  M  and  Its  data  and  assessments  of  the  training  problem.  However, 


.  •w  mi 
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In  order  to  transform  these  results  of  analysis  Into  a  practical,  workable 
plan  of  action,  additional  sources  of  Information  must  be  utilized.  Fore¬ 
most  among  these  Is  the  NAVA IR/NAVTRAEQUI PCEN  ISD  model3  which  provides 
specifications  of  the  ISO  tasks  to  be  accomplished,  their  sequencing,  and 
the  end  products  to  be  delivered  at  each  major  milestone  throughout  the 
project.  Essentially,  the  model  may  be  viewed  as  a  management  control  system 
utilized  by  the  NAVTRAEQUIPCEN  to  Insure  that  all  technical.  Instructional, 
and  logistical  requirements  will  be  accomplished  in  a  timely  coordinated 
manner.  Other  sources  of  Information  that  are  Incorporated  Into  the  PMP 
Include  the  latest  funding  analyses  available,  and  cost  data  (Navy  and 
contractor)  from  previous  ISO  projects.  Hence,  the  PMP  Is  a  docunent  which 
consolidates  the  analytical  Information  contained  in  the  PA  with  the 
decisions  for  action  derived  therefrom.  Integrates  this  dynamically  trans¬ 
formed  Informational  core  with  Inputs  from  all  other  pertinent  sources, 
and  projects  a  path  for  ISD  action  that  Is  consistent  with  the  NAVAIR/ 
NAVTRAEQUIPCEN  model,  which  may  be  used  as  the  primary  management  tool  for 
directing  a  large-scale  ISD  project.  If  the  PMP  Is  approved  by  NAVAIRSYSCOM, 
and  the  necessary  funding  Is  secured.  Phase  I  of  the  ISD  project  may  be 
Initiated. 


T.  Funaro,  J.  )r.  and  Mulligan,  B.  E.  (July  1978), "Instructional  Systems 
Design:  The  NAVAIR/ NAVTRAEQUIPCEN  Model,"  NAVTRAEQUIPCEN  Report  No.  IH-304. 
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SECTION  II 
PROBLEM  ANALYSIS 


THE  PROBLEM  ANALYSIS  REPORT 

The  contents  and  points  of  emphasis  In  a  PA  report  may  differ  depending 
on  both  the  nature  and  status  of  the  training  program.  The  following 
paragraphs  cover  the  main  Items  of  concern  In  the  evaluation  of  an  existing 
training  program  and  the  situation  in  which  the  PA  Is  most  often  applied. 
Although  the  orientation  of  a  PA  aimed  at  determining  the  training  needs  for 
a  newly  emerging  weapon  system  may  not  coincide  exactly  with  a  PA  directed 
at  assessing  the  need  for  a  revision  or  enhancement  of  an  existing  TP,  both 
are  responsible  for  providing  the  data  necessary  to  formulate  a  plan  for  TP 
development.  The  items  of  Information  described  in  the  following  paragraphs 
should,  therefore,  generally  represent  the  contents  of  a  typical  PA  report. 

THE  TRAINING  PROGRAM 

REASONS  THE  PA  WAS  CARRIED  OUT.  To  develop  a  characterization  of  the  train¬ 
ing  problem  which  may  be  viewed  from  two  perspectives,  that  existing  prior 
to  the  PA,  as  documented  In  the  IPI,  (see  p.  4),  and  that  existing  after 
completion  of  the  PA.  While  the  former  constitutes  the  review  of  the  problem 
that  ultimately  led  to  the  PA,  a  summary  of  both  views  may  not  only  provide 
an  Informative  Introduction  to  the  PA  report,  but  It  may  Indicate  the  gain 
In  understanding  of  the  problem  acquired  from  the  PA.  In  any  case,  a  concise 
statement  of  the  characteristics  of  the  training  problem,  whether  based  on  PA 
data  alone  or  combined  with  the  earlier  IPI  data,  should  serve  as  an  Intro¬ 
duction  to  the  subject  of  central  concern  In  the  PA  report. 

TIE  PROGRAM  PURPOSE 

REASONS  THE  TRAINING  PROGRAM  WAS  ORIGINALLY  INSTITUTED.  A  brief  historical 
review  of  the  conditions  and  situational  constraints  that  existed  at  the 
time  of  the  original  development  of  the  TP  In  question,  particularly  those 
factors  that  constituted  the  rationale  for  developing  and  Instituting  the 
TP.  This  section  will  be  especially  Important  If  the  original  TP  goals  (see 
section  below)  were  not  stated  explicitly,  or  If  they  cannot  be  obtained  from 
available  documentation.  The  Information  sought  here  is  not  to  be  a  replica¬ 
tion  of  that  contained  In  above  section,  the  objective  of  which  Is  to 
describe  the  nature  of  the  problem  that  arose  after  the  TP  was  Implemented. 
The  present  section  Is  concerned  with  the  context  and  factors  that  led  to  the 
original  development  of  the  training  program.  This  historical  suimary  should 
not  only  provide  a  picture  of  the  background  against  which  the  training 
problem  may  be  viewed  most  appropriately,  but  also  It  may  Indicate  what  the 
TP  was  originally  designed  to  accomplish. 

THE  PROGRAM  GOALS 

WHAT  m  TRAINING  WAS  DESIGNED  TO  ACCOMPLISH.  This  section  should  contain 
a  complete  statement  of  the  TP  goals  Including  any  revisions  and  additions 
to  the  original  goals.  In  the  absence  of  any  available  documents  containing 
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goal  statements,  it  may  be  necessary  for  the  analyst  to  deduce  the  program 
goals  from  interviews  with  instructional  personnel  and  course  syllabi. 

Even  If  documentation  of  goals  is  available,  the  analyst  should  confirm 
their  accuracy. 

There  are  several  rules  which  apply  in  the  evaluation  of  training  goals. 
First,  for  each  position  to  be  trained  there  should  be  at  least  one  goal 
statement.  Second,  each  goal  statement  should  be  derived  from  an  analysis 
of  the  task  to  be  performed  upon  completion  of  training,  and  should  Indicate 
the  level  of  proficiency  that  will  be  expected  in  the  operational  environment, 
as  well  as  the  entry-level  proficiencies,  of  new  trainees.  Third,  the  goal 
statements  for  a  given  position  should  be  such  that  they  imply,  and  are 
consistent  with,  the  behavioral  objectives  for  every  course  of  Instruction 
applicable  to  that  position  (see  CURRICULUM  ORGANIZATION).  Fourth,  for  each 
position,  there  should  be  a  goal  statement  summarizing  the  duration  of 
training  from  entry  to  exit  and  the  program's  student  through-put  capacity 
(see  page  12). 

A  generalized  goal  statement  might  read  as  follows:  for  BLANK  operator 
position,  this  training  program  is  designed  to  take  in  BLANK  number  of 
trainees  that  meet  BLANK  entry-level  requirements  and  to  train  them  in  the 
areas  of  BLANK,  BLANK,  BLANK  such  that,  at  the  end  of  BLANK  weeks,  graduates 
will  display  BLANK  attitudes  and  exhibit  BLANK  proficiencies  in  the  perform¬ 
ance  of  BLANK  tasks  under  BLANK  operational  conditions.  In  the  foregoing, 
the  word  BLANK  Indicates  the  required  data.  A  complete  enumeration  of  the 
program  goals  for  a  particular  position  will  probably  require  several  sen¬ 
tences.  In  any  case,  these  statements  should  make  explicit  what  the  training 
program  was  designed  to  accomplish,  as  opposed  to  what  the  training  program 
actually  does  accomplish. 

PROGRAM  GOALS  STATUS 

EVALUATION  OF  GOALS  AGAINST  CURRENT  AND  ANTICIPATED  NEEDS.  Make  an  appraisal 
of  training  program  goals  in  effect  at  the  time  the  PA  was  conducted,  consid¬ 
ering  the  appropriateness  of  each  goal  In  light  of  current  and  projected 
fleet  needs  and  training  support  availability.  Goals  may  require  updating 
to  reflect  alterations  In  operational  system  components,  changes  in  expected 
graduate  proficiency  levels,  stepped-up  levels  of  demand  for  program  graduates 
(or  the  inverse  of  this),  the  supply  of  students  meeting  entry-level  require¬ 
ments,  etc.  Also,  goals  may  be  out  of  line  with  the  availability  of  training 
facilities,  funds,  competent  Instructors,  etc.  If  any  of  these  factors 
represent  permanent  constraints  upon  the  TP,  then  they  should  lead  to  a  new 
goal  statement.  This  goal  statement  differs  from  that  of  the  preceding 
section  since,  here,  it  Is  the  current  and  anticipated  needs  fbr  training  that 
are  of  primary  concern  rather  than  the  needs  that  existed  at  the  time  the 
original  TP  was  designed. 

Essentially,  then,  this  section  should  contain  a  comprehensive  set  of 
explicitly  formulated  revisions  to  the  training  program  goals.  Since  these 
goals  will  best  represent  what  the  TP  can  and  should  accomplish,  they  will 
serve  as  the  criteria  for  assessing  the  conceptual  (design,  direction,  scope, 
etc.)  and  organizational  (management,  quality  control,  support  systems,  etc.) 
deficiencies  In  the  existing  TP,  as  well  as  Its  validity. 
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PROGRAM  ORGANIZATION 

RELATIONSHIP  AMONG,  AND  RESPONSIBILITIES  OF,  ALL  PARTICIPATING  UNITS.  An 
organizational  chart  that  delineates  the  formal  lines  of  authority  flowing 
from  the  highest  level  of  identifiable  responsibility  down  to  the  unlt(s) 
responsible  for  TP  Implementation.  In  the  case  of  two,  or  more,  implementing 
units  having  no  direct  organizational  ties  other  than  their  respective  contri¬ 
butions  to  the  TP,  the  structure  of  the  flow  chart  may  project  as  high  as  the 
Chief  of  Naval  Operations  (CNO)  level  before  merging.  This  overall  organiza¬ 
tional  structure  should  be  supplemented  by  an  itemization  of  the  major  respon¬ 
sibilities  assumed  by  each  command  Identified  in  the  chart.  Statements  of 
responsi bi 1 1y  areas  should  make  clear  the  mutual  commitments  for  TP  support 
(funds,  facilities,  manpower,  materials)  and  coordination  among  all  partici¬ 
pating  units. 

TRAINING  SQUADRON  ORGANIZATION 

FLEET  REPLACEMENT  SQUADRON  (FRS)  POSITIONS  AND  DUTIES.  An  organizational 
chart  that  Identifies  the  formal  lines  of  authority  within  each  squadron 
responsible  for  implementing  any  aspect  of  the  TP.  The  chart  should  begin 
with  the  commanding  officer  of  each  FRS  and  then  break  out  all  authorized 
subordinate  positions  of  responsibility.  This  detailed  organizational 
structure  should  be  supplemented  by  complete  statements  of  both  primary  and 
collateral  duties  assigned  to  each  position.  Examples  of  duty  areas  include: 
administration,  personnel  management,  resources  management,  record  maintenance 
reporting,  instruction,  instructor  training,  training  device  instruction, 
support  management,  human  relations,  safety,  and  recreation.  If  more  than 
one  squadron  Is  responsible  for  Implementing  the  TP,  then  the  respected  parts 
of  the  TP  (courses,  parts  of  courses,  etc.)  assigned  to  each  should  be 
identified,  and  any  organizational  mechanism  used  to  coordinate  activities 
of  participating  squadrons  should  be  described. 

PROGRAM  ORGANIZATIONAL  STATUS 

EVALUATION  OF  STRUCTURAL  AND  FUNCTIONAL  ASPECTS  OF  THE  TRAINING  ORGANIZATION. 
Conduct  examination  of  both  the  supporting  and  training  organizations  to 
establish  their  adequacy  for  achieving  program  goals.  Questions  of  Interest 
regarding  the  overall  organization  are:  Is  the  organization  formally  ade¬ 
quate  to  provide  the  necessary  training  support  and  coordination?  Is  the 
organization  functioning  as  intended?  Is  there  a  more  efficient  organi¬ 
zational  structure  that  is  feasible?  What  constraining  factors  would  limit 
structural  reorganization?  What  organizational  changes  would  be  beneficial 
and/or  required  In  order  to  institute  changes  In  the  TP?  Questions  of 
Interest  regarding  the  training  squadron(s)  are:  Is  the  formal  organization 
of  each  training  squadron  adequate  for  successful  Implementation  of  all 
aspects  of  the  TP?  Have  the  duties  of  FRS  personnel  been  specified  In 
sufficient  detail  to  insure  their  performance  and  to  permit  in  sufficient 
detail  to  Insure  their  performance  and  to  permit  their  evaluation?  Is  the 
FRS  organization  functioning  as  Intended?  Will  changes  In  the  TP  benefit 
by  and/or  require  reorganization  of  the  FRS?  Does  the  FRS  contain  a  training 
management  and  quality  control  system? 
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POST-GRADUATE  COMMANDS 

THE  OPERATIONAL  SQUADRONS  THAT  RECEIVED  TP  GRADUATES.  Identify  fleet 
squadrons  to  which  graduates  of  the  TP  are  assigned.  A  description  of  the 
Initial  positions  and  duties  assumed  by  TP  graduates  should  be  given  for 
each  fleet  squadron.  Questions  of  Interest  regarding  post-graduate  commands 
are:  Are  the  jobs  to  which  graduates  are  assigned  Initially  In  operational 
squadrons  commensurate  with  the  training  they  received  In  the  FRS?  Are  there 
additional  squadrons  In  the  fleet  that  could  utilize  FRS  graduates*  especially 
If  the  TP  was  enhanced?  What  directions  of  TP  enhancement  would  expand  utili¬ 
zation  of  graduates  whether  within  the  present  user  squadrons  or  within  new 
squadrons? 

INSTRUCTORS 

QUALIFICATIONS  AND  AVAILABILITY  OF  PERSONNEL  RESPONSIBLE  TOR  TRAINING.  Con¬ 
duct  a  summary  of  FRS  personnel's  capability  to  carry  out  Instructional 
requirements  of  the  TP.  The  two  areas  of  prime  Importance  to  assess  person- 
nel's  capability  are  Instructors'  qualifications  and  availability,  as  they 
exist  In  the  FRS.  Instructors'  qualifications  should  be  documented  in  the 
following  categories:  previous  experience  (operational.  Instructional,  etc.) 
that  Is  required;  rank  or  rating  requirements;  normative  selection  criteria 
(relative  test  performance);  Instructors'  training  program  requirements  and 
certifications;  Instructors'  evaluations  (types,  dimensions,  and  frequency). 
Instructors'  availability  should  be  documented  In  the  following  categories: 
the  nunber  of  qualified  Instructors  present  In  the  FRS  for  each  area 
(position)  of  training  designated  In  the  TP  syllabus;  the  netalnablllty  of 
Instructors  In  terms  of  the  normal  tour  of  duty  as  a  FRS  Instructor;  the 
nunber  of  non-lnstructlonal  personnel  In  the  FRS  that  are  qualified  to  serve 
as  Instructors  on  an  "as-needed"  basis;  the  instructional  work  loads  (nunber 
of  classes  per  day,  class  duration,  etc.)  assigned  to  instructors;  the  non- 
lnstructlonal  work  loads  (administration,  handout  preparation,  equipment 
Inventory  and  checkout,  watch  duties,  etc.)  assigned  to  Instructors;  the  use 
of  non-FRS  personnel  as  Instructors. 

STUDENT  POPULATION 

PIPELINE  SOURCES  AND  ENTRY-LEVEL  REQUIREMENTS.  Conduct  analysis  of  student 
origins,  qualifications,  and  program  entry-level  requirements  for  each 
position  to  be  trained.  Identify  the  sequential  flow  of  students  through 
the  various  command  units  (pipeline)  that  ultimately  feed  Into  the  TP  to 
provide  data  on  the  previous  training  received  and  qualifications  attained 
by  students,  as  well  as  the  data  necessary  to  assess  problems  related  to 
quota  control.  Likewise,  any  entry-level  requirements  (prerequisite  training 
programs,  prior  aircraft  experience,  NATOPS  qualifications,  service  group 
rating,  security  clearance,  etc.)  that  are  operative  In  the  selection  of 
students  for  adnlsslon  to  the  TP  should  be  listed  for  each  position  to  be 
trained.  These  data  not  only  help  to  define  the  general  characteristics 
of  the  student  population  for  which  the  TP  presumably  was  designed,  but  they 
may  also  enumerate  detailed  characteristics  of  the  student  population  that 
exert  a  selective  Influence  on  the  success  of  the  TP,  e.g.,  specific 
student  readiness  deficiencies. 
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Questions  of  Interest  regarding  composition  of  the  student  population 
are:  Does  the  TP  require  training  In  any  area  for  which  entering  students 
are  unprepared.  Improperly  prepared  (proactive  Inhibition),  or  already 
fully  competent?  In  the  case  of  essential  entry-level  skills  and/or  knowl¬ 
edges  that  are  either  lacking  or  at  variance  with  those  to  be  required  In 
the  present  TP,  will  special  media  (trainers,  computer  programs,  etc.)  or 
additions  to  the  syllabus  be  required  In  order  to  remedy  these  deficiencies? 

If  special  media  are  required,  will  they  have  to  be  developed,  or  can  they 
be  made  available  through  arrangements  with  other  commands?  Can  entry-level 
deficiencies  be  eliminated  by  a  reorganization  of  the  pipeline?  Do  entry- 
level  deficiencies  contribute  to  poor  student  motivation  and  attitudes.  If 
such  exists?  Can  entry-level  requirements  of  the  TP  be  revised  so  as  to 
reduce  entry-level  deficiencies? 

STUDENT  THROUGH -PUT 

CLASS  LOADS,  PROGRAM  CAPACITY,  AND  STUDENT  FLOW.  Make  analysis  of  student 
through-put  and  program  capacity  for  each  position  to  be  trained,  and  each 
category  (courses,  etc.)  and  phase  of  the  annual  program  cycle.  The  data 
may  be  summarized  conveniently  In  terms  of  a  yearly  schedule  of  classes. 

Such  a  schedule  would  Identify  each  class  by  Its  proper  designator  and  start- 
completion  dates,  and  thus  provide  a  framework  In  which  student  through-put 
data  may  be  entered.  The  actual  nunber  of  students  processed  and  the 
maximum  nunber  of  students  that  can  be  effectively  processed  are  the  two  data 
Items  that  should  be  given  for  each  offering  of  each  class.  The  first  datum 
should  be  a  matter  of  record.  The  second  should  be  estimated  on  the  basis 
of  the  following: 

a.  Availability  of  adequate  training  facilities  (see  PHYSICAL  RESOURCES) 

b.  Availability  of  proper  Instructional  materials  and  media  (see 
PHYSICAL  RESOURCES) 

c.  Instructor-to-student  ratio  (see  INSTRUCTORS). 

The  maximum  number  of  students  that  can  be  effectively  processed  per  class 
can  be  taken  as  an  Index  of  the  effective  capacity  of  each  class  offered. 

This  schedule  analysis  may  be  supplemented  by  an  overall  flow  diagram  which 
would  show  the  paths  (training  tracks)  specified  In  the  syllabus  fbr  all 
positions. 

Questions  of  Interest  regarding  student  through-put  are:  What  Is  the 
magnitude  of  the  TP  In  terms  of  actual  nunbers?  Does  actual  through-put 
exceed  or  fall  short  of  the  program's  effective  capacity  In  any  area  of 
training?  Is  effective  capacity  too  low  In  any  area  of  the  TP  due  to  a  lack 
of  facilities,  materials,  media,  and/or  Instructors?  If  through-put  exceeds 
effective  capacity  In  a  given  area,  what  are  the  consequences  of  this  Interns 
of  student  performance?  Has  through-put  fallen  below  the  original  TP  goals 
(see  PROGRAM  GOALS)  In  sane  program  area  as  a  result  of  a  low  effective 
capacity?  Have  student  attrition  rates  been  excessive  In  any  particular 
program  area?  Is  the  effective  capacity  In  all  areas  of  the  program 
sufficient  to  meet  newly  evolved  TP  goals?  Is  the  flow  of  students  through 
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the  syllabus  consistent  with  the  hierarchy  of  behavioral  objectives  so  that 
the  objectives  Intermediate  between  entry-level  and  final  task  performance 
and  students  are  trained  In  the  proper  progression?  Are  "hand-on"  experi¬ 
ences  programmed  so  as  to  reenforce  academic  training?  In  the  case  of  tasks 
specified  for  "deferred  training,"  are  provisions  for  this  adequate  to 
achieve  the  necessary  training  at  the  post-FRS  duty  stations?  If  the  TP 
does  not  operate  on  a  scheduled  basis  with  regular  start-completion  dates.  Is 
the  TP  offered  on  sane  other  basis,  e.g.,  "as-needed"?  Is  this  basis  adequate 
to  meet  program  goals?  Are  any  alterations  of,  or  additions  to  the  syllabus 
required  (see  CURRICULUM  ORGANIZATION)?  Can  syllabus  changes  be  avoided  by 
reducing  either  the  rate  of  student  through-put  (e.g.,  an  Increase  in  course 
duration),  or  the  magnitude  of  through-put  relative  to  program  effective 
capacity? 

S1UDENTS  PROGRESS 

PERFORMANCE,  MOTIVATION,  AFC  ATTITUDES  OF  STUDENTS  TRACKED  THROUGH  THE  TP. 
Conduct  analysis  aimed  at  determining  the  degree  to  which  the  Instructional 
objectives  are  being  achieved.  It  Is  here  that  the  Instructional  effect¬ 
iveness  of  the  TP  is  assessed.  The  data  to  be  collected  should.  If  it  is 
available,  answer  the  following  question:  Does  student  performance  change 
after  exposure  to  the  TP  such  that  expected  program  achievement  levels  are 
met?  The  answer  to  this  question  should  be  documented  not  only  in  terms  of 
performance  relative  to  criterion -referenced  measures  for  behavioral  objec¬ 
tive  encompassed  by  the  TP,  but  also  In  terms  of  motivation  and  attitude 
measures. 

In  order  to  assure  that  observed  changes  in  the  various  student  measures 
are  attributable  solely  to  the  Instructional  effects  of  the  TP,  several  pre¬ 
cautions  will  be  helpful. 

Assailing  that  performance,  etc.,  is  tracked  from  just  before  training 
begins  (pre-test),  throughout  training  (lesson  tests),  and  finally  to  comple¬ 
tion  of  training  (possibly  an  "on-the-job"  test),  the  analyst  must  account 
for  any  events  that  may  occur  between  tests  (a  change  in  the  grading  system, 
instronentatlon  breakdown,  pay  Increase,  etc.)  that  may  account  for  the 
observed  change  In  test  performance.  In  addition  to  specific  events,  the 
analyst  should  look  for  temporal  trends  In  test  performance  that  may  be  the 
result  of  such  gradual  Influences  as  time  away  from  family,  boredom,  and 
experience  In  the  training  setting  (e.g.,  habitual  distracting  features). 
Further,  the  analyst  should  make  sure  that  a  general  trend  toward  Improved 
test  performance  Is  not  due  to  experience  gained  on  successive  tests,  a 
cumulative  effect  unrelated  to  Instruction. 

In  the  case  of  programs  that  allow  students  to  progress  from  one  stage 
to  another  only  If  their  performance  exceeds  some  high  cutoff  level,  there 
Is  a  non-lnstructlonal  effect  which  may  result  In  an  apparent  decrease  In 
performance  on  the  test  at  the  next  stage.  This  Is  known  as  statistical 
regression  toward  the  mean,  and  It  is  the  result  of  selecting  students  to 
advance  only  If  they  score  above  some  cutoff  at  the  extreme  end  of  the 
performance  distribution.  The  reverse  effect  may  occur  as  a  result  of 
attrition,  l.e..  If  low-scoring  students  drop  out  of  the  program,  then 
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performance  on  subsequent  tests  by  students  who  continue  in  the  program  may 
appear  to  Improve. 

By  taking  these  factors  Into  account,  the  analyst  will  help  to  Insure 
that  non-lnstructional  effects  are  not  attributed  erroneously  to  Instruction. 
Ihls  pitfall  may  be  avoided  more  substantially  by  means  of  a  control  group 
that  Is  treated  In  exactly  the  same  way  as  the  student  group  In  all  respects 
except  that  the  control  group  does  not  participate  In  the  TP  and,  hence,  can¬ 
not  benefit  from  It.  If  the  control  group  approach  Is  feasible,  care  should 
be  taken  to  Insure  that  both  the  students  and  controls  were  drawn  from  the 
same  population.  An  optimal  procedure  Is  to  match  individuals  on  all  para¬ 
meters  of  Importance,  and  then  to  assign  them  randomly  to  each  group.  Al¬ 
though  a  control  group  will  probably  not  be  possible  in  most  established 
situations.  It  may  be  possible  to  obtain  some  comparable  data  relating  student 
performance  In  the  TP  under  consideration  with  that  of  Individuals  trained 
under  a  different  program.  This  data  will  probably  be  "on-the-job*1  perform¬ 
ance  evaluations  and  considerable  caution  should  be  exercised  In  drawing 
conclusions  from  such  comparisons. 

A  related  area  of  concern  Involves  programs  that  originally  were 
subjected  to  a  tryout-revision  process.  If  the  TP  effectiveness  Is  found  to 
fall  short  of  the  expectations  established  during  tryouts,  the  analyst  should 
determine  whether  this  Is  due  to  such  forces  as  differences  between  the 
current  student  population  and  the  population  used  as  a  source  of  tryout 
personnel,  differences  between  the  tryout  situation  and  the  training  situ¬ 
ation,  and  differences  between  the  treatments  (higher  levels  of  motivation, 
etc.)  received  by  Individuals  who  participated  In  tryouts  as  opposed  to  those 
In  the  TP. 

The  foregoing  Is  based  on  the  assumption  that  the  TP  has  Identifiable, 

If  not  specifically  stated,  behavioral  objectives  that  not  only  stipulate  the 
actions  to  be  performed,  but  also  the  conditions  of  performance  and  the 
standards  below  which  performance  Is  judged  to  be  unacceptable.  In  the 
absence  of  such  objectives,  the  analyst  will  have  little  basis  for  determining 
appropriate  performance  criteria,  and  hence  for  evaluating  the  Instructional 
effectiveness  of  the  training  program.  In  the  case  of  a  TP  with  no  stated 
behavioral  objectives,  the  analyst  should  attempt  to  deduce  the  actual 
criteria  Implied  In  performance  tests  and  to  determine  whether  they  provide 
a  valid  and  unbiased  estimate  of  the  performance  to  be  expected  on-the-job. 

A  TP  cannot  be  considered  effective  If  It  does  not  result  In  an  Improved 
job  performance,  even  If  It  does  succeed  In  Improving  performance  on  tests 
that  are  Invalid.  In  any  case,  the  analyst  should  consider  In  detail  both 
the  methods  of  testing  (test  type)  and  the  methods  of  evaluating  test 
results,  as  well  as  the  subject  matter  tested,  the  testing  schedule,  the 
system  for  providing  test  results  to  students,  the  system  employed  for  re¬ 
cording  and  storing  test  results,  and  the  procedures  used  In  tracking  and 
evaluating  student  progress.  It  Is  also  Important  to  determine  whether  the 
scope  and  depth  of  the  Information  actually  taught  Is  adequately  represented 
In  tests.  Further,  the  manner  In  which  test  results  are  consolidated  and  used 
to  establish  final  evaluations  of  student  performance  should  be  carefully 
examined. 
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CURRICULUM  ORGANIZATION 

TRAINING  TRACKS,  COURSE  SYLLABI,  AM)  BEHAVIORAL  OBJECTIVES.  An  analysis  of 
the  sequential  organization  of  courses  and  course  oontents.  It  Is  convenient 
to  divide  this  analysis  Into  two  parts:  (1)  training  tracks,  and  (2)  course 
syllabi  and  objectives.  The  former  Is  a  comprehensive  map  which  shows  the 
ordinal  arrangement  of  courses  for  each  position  to  be  trained  (see  STUDENT 
THROUGH-PUT).  The  latter  Is  a  breakdown  of  each  course  into  Its  component 
units,  lessons,  segments,  and  "hands-on"  exercises  with  statements  of  the 
behavioral  objectives  to  be  achieved  within  each  component  part. 

The  map  Illustrating  program  tracks  provides  an  overall  view  of  the 
organization  of  Instructional  content.  For  each  position  to  be  trained, 
the  order  In  which  a  student  successively  encounters  each  general  content 
area  may  be  traced  from  program  entry  to  exit.  The  general  cohesiveness  of 
the  content  organization  within  tracks  may  be  made  visible  by  means  of  such 
a  map.  Also,  courses  that  are  cornnon  to  more  than  one  track,  and  which  may 
have  been  duplicated,  or  even  onmltted,  may  be  Identified.  Further,  such  a 
map  can  show  the  points  In  the  Instructional  program  where  coordination  of 
instructional  Inputs  to  the  program  from  other  squadrons  should  occur  In 
cases  where  more  than  one  unit  Is  responsible  for  training. 

The  second  part  of  this  analysis  Is  an  assessment  of  the  Instructional 
contents  of  courses  In  light  of  program  goals  and  course  objectives.  Theo¬ 
retically,  If  all  the  behavioral  objectives  for  every  course  of  Instruction 
that  are  Included  In  the  training  track  for  a  given  position  are  combined, 
they  should  provide  a  complete  and  accurate  specification  of  the  program's 
Instructional  goals  for  that  position.  While  this  theoretically  Ideal  situ¬ 
ation  might  never  be  found  to  exist.  It  Indicates  the  kind  of  relationship 
that  should  exist  between  program  goals  and  course  objectives.  If  the 
program  goals  for  a  given  position  have  been  formulated  to  accurately  encom¬ 
pass  the  tasks  to  be  performed,  then  these  goals  may  serve  as  a  basis  for 
evaluating  course  objectives.  If  the  behavioral  objectives  for  all  courses 
In  a  training  track  are  found  to  be  Incomplete  relative  to  the  goals  of  the 
program,  then  It  Is  likely  that  either  course  enhancement  or  addition  Is  re¬ 
quired.  However,  If  the  behavioral  objectives  are  found  to  be  excessive 
relative  to  program  goals.  It  Is  probable  that  either  course  revision  or 
deletion  Is  needed.  An  analogous  comparison  should  be  made  between  course 
objectives  and  lesson  objectives. 

The  foregoing  analysis  establishes  the  completeness  and  appropriateness 
of  course  contents,  but  It  does  not  access  the  Integral  nature  of  the  con¬ 
tents.  A  well-designed  sequence  of  Instructional  divisions  (courses,  wits, 
lessons,  lesson  segments,  and  "hands-on" exercises)  Insures  that  students 
possess  just  those  requisite  skills  and  knowledges  at  each  stage  of  training 
that  are  necessary  for  them  to  move  successfully  on  to  the  next  stage.  This 
Is  usually  accomplished  by  sequencing  elemental  objectives  before  more 
advanced  ones.  A  careful  examination  of  the  syllabus  for  each  course  should 
reveal  any  gaps  or  sequential  misalignments  of  course  contents.  As  a  general 
rule,  the  following  organization  of  course  contents  has  been  Ibund  to  be  the 
most  useful. 
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Entry-levels  are  taken  as  Indicators  of  how  elementary  the  first  object¬ 
ives  must  be.  These  are  divided  into  segments  which  ere  then  grouped  to  form 
lessons  that  are  followed  by  tests  (see  STUDENT  PROGRESS).  Lessons  are 
organized  into  instructional  units  which  are  followed  by  "hands-on"  exercises 
that  allow  students  to  practice  that  which  has  been  learned  at  each  stage 
before  they  move  on  to  other  subject  matter,  whether  it  be  elemental  or  more 
advanced.  The  syllabus  may  cycle  students  between  a  number  of  elementary 
academic  units  and  corresponding  "hands-on"  exercises  before  proceeding  to 
the  next  level  of  training.  However,  Instructional  units  should  be  formed 
so  as  to  permit  practice  and  application  of  principles  before  they  are  for¬ 
gotten  or  confused  with  others.  As  the  units  become  progressively  more 
advanced,  the  exercises  thar  are  required  should  also  Increase  In  difficulty, 
but  without  exceeding  that  which  has  been  learned  academically.  Initial 
"hands-on"  exercises  may  be  at  the  famllarizatlonal  level,  while  subsequent 
exercises  Involve  practice  In  simulators  and,  eventually.  In  the  operational 
environment. 

The  data  that  Is  necessary  to  perform  a  curriculum  analysis  consists  of 
complete  syllabi  for  all  courses  and  exercises  (trainers,  equipment,  and  air¬ 
craft).  It  will  be  convenient  to  attach  these  documents,  if  they  are  avail¬ 
able,  as  appendices  to  the  PA  report.  From  these  documents,  as  well  as  other 
sources,  the  analyst  may  abstract  the  training  track  map,  the  program  goals, 
the  course  objectives  from  units  down  to  lesson  segments,  and  the  sequential 
organization  of  Instructional  divisions  within  each  course.  From  this  data 
base.  It  should  be  possible  to  evaluate  the  organizational  structure  of  the 
curriculum  as  well  as  the  Instructional  completeness  and  Integrity  of  Indivi¬ 
dual  courses.  Also,  It  should  be  possible  to  determine  whether  courses  are 
designed  to  meet  program  goals  and,  additionally,  to  Identify  missing,  weak, 
duplicated,  or  misdirected  courses.  This  data  also  may  cast  light  on  the 
source  of  curriculum  problems,  e.g.,  students  required  by  logistical  con¬ 
straints  to  deal  with  subject  matter  at  a  level  for  which  they  lack  the 
requisite  skills. 

PHYSICAL  RESOURCES 

INVENTORY  AND  EVALUATION  OF  INSTRUCTIONAL  MEDIA  AND  FACILITIES.  Make  analysis 
of  the  physical  capacity  of  the  TP  to  fulfill  course  objectives.  An  Inven¬ 
tory  of  Instructional  media  and  facilities  for  each  area  of  training  Is 
necessary  to  establish  the  existing  availability  of  resources,  but  this 
Inventory  alone  will  not  Indicate  the  need  for  certain  media  and  facilities 
that  may  exist  In  aqy  particular  training  category.  Both  availability  and 
need  should  be  documented  In  detail  and,  from  this  data,  a  physical  resources 
requirement  be  determined.  Specific  media  and  facility  needs  should  be 
established  for  each  course  and  training  track  taking  Into  account  student 
through-put  data  and  the  nature  of  Instruction  Indicated  In  the  behavioral 
objectives  for  each  lesson  and  exercise  (see  CURRICULUM  ORGANIZATION).  The 
following  media  and  facilities  categories  should  be  Inventoried  for  both 
availability  and  need. 

a.  Media: 

(1)  Training  equipment  (projectors,  recorders,  screens,  etc.) 
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(2)  Maintenance  and  production  equipment,  tools,  and  supplies 

(3)  Media  software  (slides,  workbooks,  tests,  etc.) 

(4)  Trainers 

(5)  Aircraft  (aircraft  par  sortla) 
b.  Facilities: 

(1)  Classrooms  (number,  dimensions,  lighting,  acoustics,  etc.) 

(2)  Office  space  for  Instructors  and  staff 

(3)  Furniture  (classroom,  office,  etc.) 

(4)  Space  for  media  storage,  maintenance,  and  production 

(5)  Student  learning  center  and  study  areas 

(6)  Computer  terminal  area 

(7)  Avionics  test  bench  areas 

(8)  Ramp  and  hanger  spaces. 

Questions  of  Interest  are:  What  media  are  needed  to  achieve  the  behav¬ 
ioral  objectives  fbr  each  course?  Do  available  media  meet  specifications 
derived  from  course  objectives?  Are  all  available  media  operational?  Are 
facilities  and  equipment  In  the  training  squadron  adequate  to  keep  redla  In 
good  repair  througout  the  lifetime  of  the  program?  If  not,  are  there  other 
units  available  for  maintenance  activities?  Does  the  training  squadron 
possess  the  equipment,  facilities,  and  expertise  for  updating,  revising,  and 
producing  new  media  software?  If  not,  do  alternative  strategies  exist  that 
may  be  anployed  for  this  service?  Are  media  hardware  compatible  with  soft- 
nare?  Do  trainers  and  available  aircraft  possess  the  necessary  systems  fbr 
training?  Are  classrooms  adequate  to  meet  student  needs  and  pregram  sche- 
diles?  Are  study  areas.  Individualized  Instruction  areas,  etc.,  adequate? 

Is  space  available  for  a  properly  managed  media  library?  Is  the  office 
space  available  to  Instructors  and  program  managers  adequate?  Are  there 
other  facilities  that  may  be  converted  to  use  of  the  training  squadron?  Are 
there  any  Items  of  equipment,  etc.,  on  order  which  are  not  reflected  In  the 
Inventory?  Are  all  facilities  located  within  an  area  sufficiently  mail  to 
penult  easy  movement  from  one  place  to  another?  If  not,  are  there  sched¬ 
uling  problens,  transportation  problems,  etc.,  that  exert  a  detrimental 
Influence  on  the  success  of  Instruction? 

CONCLUSIONS  AM)  RECOMMENDATIONS 


SUMMARY  OF  MAJOR  FINDINGS  NECESSARY  FOR  PROJECT  PLANNING.  The  contents  of 
the  PA  report  serve  as  the  primary  data  source  fbr  development  of  the 
project  master  plan  (PMP).  Whether  a  revision,  an  enhancement,  or  an  en¬ 
tirely  new  TP  Is  required,  the  PMP  Is  the  plan  that  will  govern  the 
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necessary  nanagerlal  and  technical  aspects  of  TP  development,  production. 
Implementation,  and  lono-term  support.  The  success  of  this  plan  Is  heavily 
dependent  upon  the  completeness  and  utlllzablllty  of  the  Information  provided 
in  the  PA  report.  Assuming  that  the  analyses  Indicated  In  previous  sections 
have  been  carried  out  fully,  attention  should  now  be  directed  to  a  suirmary 
organization  of  this  Information  that  will  make  It  maximally  usable  during 
project  planning.  The  precise  nature  pf  the  training  problem  should  be 
delineated  together  with  recomnendatlons  for  Its  remedy.  The  planners  should 
be  able  to  see  which  aspects  of  the  existing  TP  contribute  to  the  problem  and 
be  able  to  understand  the  overall  context  In  which  the  problem  Is  manifest. 
Alternative  strategies  for  solving  the  various  components  of  the  problem 
should  be  offered  whenever  possible.  In  brief,  the  conclusions  and  recommend 
atlons  called  fbr  In  this  section  are  those  that  will  be  essential  to  the 
development  of  a  plan  to  alleviate  the  training  problem. 
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SECTION  III 
PROJECT  MASTER  PLAN 

This  is  the  primary  work  plan  which  Kill  guide  and  coordinate  develop¬ 
ment  of  the  new  training  program,  or  the  revisions  to  be  Incorporated  Into 
an  existing  TP.  Utilizing  the  Information  provided  In  the  PA,  together  with 
other  Inputs  such  as  ISO  specifications  (MIL-T-29053)  and  associated  Data 
Item  Descriptions  (DIDs),  the  NAVTRAEQUIPCEN  Initiates  a  course  of  actio 
which  ultimately  results  In  the  PMP.  This  plan  coordinates  every  aspect 
of  the  project  Including  Its  development,  production.  Implementation,  and 
long-term  support.  The  PMP  must  Insure  efficient  utilization  of  available 
resources,  adequate  support  throughout  the  life-cycle  of  the  TP,  and  continued 
instructional  effectiveness  and  technical  currency  through  a  system  of  moni¬ 
tored  performance  and  quality  control. 

PMP  OBJECTIVES 

Considering  the  diversity  of  organizations,  technical  expertise,  and 
resources  required  to  successfully  deliver  and  put  Into  operation  training 
systems  of  the  magnitude  and  complexity  of  those  current  In  Naval  aviation. 

It  Is  evident  that  a  plan  to  accomplish  this  will  have  to  be  correspondingly 
broad  In  scope.  Essentially,  such  a  plan  must  be  a  management  tool  that 
coordinates  the  contributions  of  all  participants  to  the  project  and  which 
monitors  the  timeliness  and  quality  of  each  contribution.  A  plan  that 
possesses  these  characteristics  will  have  to  achieve  the  following  objectives: 

a.  Review  training  program  goals,  establish  current  training  defici¬ 
encies  and  projected  needs,  and  specify  the  training  materials,  media,  facili¬ 
ties,  and  management  systems  to  be  developed  within  the  arena  of  this  project. 

b.  Identify  the  organizations,  agencies,  and  offices  that  will  support 
development  of  the  training  system,  and  define  the  roles,  responsibilities, 
and  resource  commitments  of  each  group. 

c.  Identify  the  committees  and  teams  that  will  provide  either  profes¬ 
sional  or  technical  support  for  development  of  the  training  system,  and 
determine  for  each  group  the  tasks  to  be  performed  and  the  products  to  be 
delivered. 

d.  Establish  a  system  for  liaison,  coordination,  and  communication 
among  all  groups  and  organizations  participating  in  the  training  system 
development. 

e.  Construct  a  time-based  schedule  of  project  activities  that  specify 
the  start  and  completion  dates  for  major  tasks,  the  delivery  dates  for 
training  products,  and  the  time  periods  that  will  be  available  for  review 
and  evaluation  of  deliverables. 

f.  Given  funding,  resources,  and  time  constraints,  develop  training 
system  specifications  (ISO  specifications,  training  device  specifications, 
etc.)  and  procurement  procedures  that  will  Insure  the  acquisition  of  an 
efficient  and  cost/effective  training  system  and  Its  successful  Implementa¬ 
tion. 


( 
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PA  IKPUTS  TO  PROJECT  PLAN 

It  would  seem  logical  to  assume  that,  for  purposes  of  developing  the 
project  master  plan,  the  essential  Information  contained  In  the  PA  report 
would  be  the  characterization  of  the  existing  training  problem  (see  TRAINING 
PROGRAM).  While  this  Information  undoubtedly  serves  to  focus  the  planner's 
attention  on  the  more  troublesome  aspects  of  the  program,  an  examination 
of  the  PMP  objectives  Indicates  that  the  entire  contents  of  the  PA  report 
will  be  necessary  In  order  to  achieve  these  objectives.  The  sections  dealing 
with  training  goals,  curricula,  and  physical  resources  provide  the  primary 
Information  necessary  to  meet  Objective  I,  although  Important  data  that  also 
relate  to  this  objective  are  contained  In  other  sections.  Objectives  II,  III, 
and  IV  depend  heavily  on  the  sections  concerned  with  program  organization, 
training  squadron  organization,  and  post-graduate  commands.  To  meet  Object¬ 
ive  VI,  planners  must  utilize  the  Information  contained  In  the  section  on 
physical  resources,  but  they  must  also  consult  the  sections  on  student 
population,  and  student  through-put.  For  all  objectives,  planners  require 
Infbrmatlon  from  sources  other  than  the  PA  report  (ISO  specifications,  funding 
analyses,  equipment  specifications,  procurement  procedures,  etc.),  especially 
In  the  case  of  Objective  V.  However,  It  Is  the  PA  report  that  provides  the 
Informational  core  upon  which  the  planning  process  Is  based,  and  this  core 
Is  presented  In  summary  form  In  the  section  devoted  to  conclusions  and 
recommendations. 

POST  PMP  USES  OF  Ttf  PA  REPORT 

In  addition  to  its  value  as  a  data  source,  the  PA  report  Is  also  an  eval¬ 
uative  document.  Throughout  Its  various  sections,  the  PA  report  contains 
both  detailed  analyses  and  evaluations  of  specific  aspects  of  the  training 
program.  Consequently,  If  the  PMP  calls  for  ISO  revision  of  the  training 
system,  much  of  the  work  will  have  been  completed  already  In  the  PA.  The 
task  analysis  stage  of  ISD  may  be  bypassed  and  the  appropriate  part  of  the 
PA  substituted  In  Its  place.  Likewise,  the  analyses  that  would  otherwise  have 
been  necessary  to  determine  program  goals,  entry-levels,  behavioral  objective* 
media  and  facilities  requirements,  etc.,  may  be  considerably  reduced.  Espec¬ 
ially  Important  In  the  case  of  programs  to  be  revised  or  enhanced  are  the 
requirements  that  need  to  be  met  to  Insure  that  the  newly  developed  components 
of  the  training  and  management  system  will  dovetail  with  the  unchanged  parts 
of  the  existing  program.  These  requirements  would  have  been  evident  from  the 
outset  In  the  PA  report. 
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